To analyse visual systems, the concept of an ideal observer promises an optimal response for a given task. Bayesian ideal observers can provide optimal responses under uncertainty, if they are given the true distributions as input. In visual search tasks, prior studies have used signal to noise ratio (SNR) or psychophysics experiments to set the distributional parameters for simple targets on backgrounds with known patterns, however these methods do not easily translate to complex targets on natural scenes. Here, we develop a model of target detectability in natural images to estimate the parameters of target-present and target-absent distributions for a visual search task. We present a novel approach for approximating the foveated detectability of a known target in natural backgrounds based on biological aspects of human visual system. Our model considers both the uncertainty about target position and the visual system's variability due to its reduced performance in the periphery compared to the fovea. Our automated prediction algorithm uses trained logistic regression as a post processing phase of a pre-trained deep neural network. Eye tracking data from 12 observers detecting targets on natural image backgrounds are used as ground truth to tune foveation parameters and evaluate the model, using cross-validation. Finally, the model of target detectability is used in a Bayesian ideal observer model of visual search, and compared to human search performance.

This paper presents a method to measure target detectability in natural images. It provides a visual search model (based on extracted features of a pre-trained CNN) to perform target detectability as a function of retinal eccentricity for human vision. Reviewers, including myself, appreciate that this paper tackles a topic that has not been well investigated in the visual search literature. The approach is well-motivated and paper is well written, and comparison with human data is a nice validation of the approach. There were issues concerning correctness of the approach, along with minor points, but the author's rebuttal has done an adequate job in addressing the concerns and I expect to see the camera ready version of the paper incorporate improvements to at will improve the clarity of the paper (esp with regards to reviewer's main concerns) using the extra page. I think this will be a nice addition to the NeurIPS2020 conference encouraging the community to look at a fresh topic, so I'm going to recommend we accept this work as a poster.

Additional Feedback: The sentence (line 123-124) "We extract features from the last fully connected layer of the CNN (before the classification layer) to use as the input to our classifier" is circular. The caption for figure 2 mentions both "detectability" and "discriminability", but only discriminability is shown. The associated text does not provide a definition of detectability. A definition is only given for discriminability. Is it the case (as seems to be implied by its usage in section 3.1.2) If they are actually different, then a clear definition needs to be given for detectability.

To analyse visual systems, the concept of an ideal observer promises an optimal response for a given task. Bayesian ideal observers can provide optimal responses under uncertainty, if they are given the true distributions as input. In visual search tasks, prior studies have used signal to noise ratio (SNR) or psychophysics experiments to set the distributional parameters for simple targets on backgrounds with known patterns, however these methods do not easily translate to complex targets on natural scenes. Here, we develop a model of target detectability in natural images to estimate the parameters of target-present and target-absent distributions for a visual search task. We present a novel approach for approximating the foveated detectability of a known target in natural backgrounds based on biological aspects of human visual system.